DEDIS: Exact Deduplication for Primary Distributed Storage∗

The removal of duplicate data from primary storage volumes in a cloud computing environment is increasingly desirable, as the resulting space savings contribute to the cost effectiveness of a large scale multi-tenant infrastructure. However, traditional archival and backup deduplication systems are not suited for large scale virtualized infrastructures and the I/O demanding applications there deployed. In fact, few deduplication systems address primary data and are either restricted to special file systems or centralized designs. This paper presents DEDIS, a dependable and fully distributed deduplication system for large-scale cloud infrastructures with a common primary storage pool abstraction. Unlike previous proposals, DEDIS does not depend on specific file systems with built-in aliasing operations while ensuring low storage I/O overhead, as described in our preliminary evaluation results. In more detail, DEDIS is implemented within Xen as a blktap driver and performs off-line deduplication among remote virtual machines.